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The  Department  of  Defense  (dod)  uses  a  relative  risk  site  evaluation 
process  as  part  of  its  decision  criteria  to  allocate  about  $2  billion  annually 
to  clean  up  contaminated  sites  that  pose  the  greatest  risk  to  human  health 
and  the  environment,  dod  estimates  that  it  will  spend  about  $27  billion  to 
complete  cleanup  on  contaminated  sites  from  fiscal  year  1998  to 
completion. 

The  Senate  Report  on  the  National  Defense  Authorization  Act  for  Fiscal 
Year  1998  (Report  105-29,  Jime  17, 1997)  requires  us  to  review  dod’s 
relative  risk  site  evaluation  process.  In  response  to  that  requirement,  this 
report  addresses  (1)  the  extent  to  which  dod  has  issued  uniform  relative 
risk  site  evaluation  guidance  and  the  application  of  the  guidance  by  the 
defense  components  and  (2)  whether  the  relative  risk  site  evaluation 
process  provided  data  that  enabled  the  defense  components  to  categorize 
sites  and  prioritize  required  work.  This  report  complements  the 
information  previoiasly  provided  to  you  on  dod’s  relative  risk  site 
evaluation  process.^ 


Background 


DOD  adopted  the  relative  risk  site  evaluation  process  in  1994  to  address 
inconsistencies  in  the  evaluation  methods  it  used  to  prioritize 
contaminated  sites.  The  process  is  intended  to  provide  defense 
components  with  a  common  methodology  for  assigning  high,  medium,  and 
low  relative  risk  categorizations  at  each  potentially  contaminated  site  on 
the  basis  of  evaluations  of  water,  soil,  and  sediments  for  their 
contamination  levels;  the  likelihood  of  contaminant  migration;  and  the 
presence  of  potential  receptors  such  as  humans,  plants,  and  animals.  In 
addition,  dod’s  relative  risk  site  evaluation  guidance  requires  that  sites 
lacking  sufficient  information  for  a  relative  risk  site  evaluation  be  given  a 
“not  evaluated”  designation,  and  provides  that  certain  other  sites  do  not 


‘Enviroirmental  Protection:  Information  Used  for  Defense  Environmental  Mant^ement 
(GAO/NSIAD-97“135,  June  11, 1997)  and  Environmental  Cleanup:  DOD’s  Relative  Risk  Process 
(GAO/NSIAD-98-79R,  Feb.  26, 1998), 
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require  evaluation.^  Not  evaluated  sites,  sites  that  do  not  require 
evaluation,  and  sites  with  risk  characterizations  are  reported  in  the 
Defense  Environmental  Restoration  Program  annual  reports  and  budget 
justification  exhibits  provided  to  Congress. 

Environmental  remediation  includes  cleanup  and  other  efforts  aimed  at 
reducing  the  risk  to  an  acceptable  level.  The  Comprehensive 
Environmental  Response,  Compensation,  and  Liability  Act  of  1980,  as 
amended,  serves  as  the  statutory  basis  for  the  environmental  remediation 
of  contaminated  sites.  Under  the  act,  the  Environmental  Protection 
Agency  (epa)  ranks  sites  for  inclusion  on  the  National  Priorities  List  on  the 
basis  of  public  health  risks  and  other  factors.®  dod’s  relative  risk  site 
evaluation  process  provides  a  tool  for  categorizing  sites  and  sequencing 
priorities  for  cleanup  on  the  basis  of  relative  risk.  In  the  relative  risk  site 
evaluation  process,  relative  risk  site  evaluations  are  used  only  to  screen 
and  categorize  sites  and  are  not  substitutes  for  baseline  risk  assessments.^ 

DOD  has  stated  that  the  relative  risk  site  evaluation  process  provides  a 
quantifiable  basis  for  justifying  requirements  and  allocating  funds.  It  has 
also  stated  tiiat  relative  risk  is  only  one  of  the  priority-setting  factors 
managers  consider  for  programming  and  sequencing  work  at  and  across 
defense  installations.  Also  important,  according  to  dod,  are  such  things  as 
program  and  economic  evaluations  and  baseline  risk  assessments  that 
managers  may  use  to  determine  which  sites  should  be  worked  on  first  in 
light  of  available  resoiuces.® 

Since  dod’s  implementation  of  the  process  in  1994,  Congress  has  raised 
questions  about  the  relative  risk  site  evaluation  methodology.  In 


^The  guidance  requires  sites  designated  as  not  evaluated  to  be  programmed  for  additional  study,  a 
removal  action  if  warranted,  or  other  appropriate  response  action,  including  deferral,  before  they  are 
evaluated.  DOD  does  not  require  relative  risk  site  evaluations  for  sites  that  require  only  the  removal  of 
building  demolition/debris  or  contain  abandoned  ordnance,  have  final  cleanup  remedies  in  place,  or 
are  designated  as  “response  complete” — ^require  no  further  cleanup  action.  A  risk-based  priority 
system  is  being  developed  for  unexploded  ordnance  removal. 

®The  National  Priorities  List  is  the  list  of  the  nation’s  most  heavily  contaminated  sites. 

^DOD’s  1997  updated  Relative  Risk  Site  Evaluation  Primer  and  subsequent  March  1998  guidance 
amplify  this  policy.  Under  the  act,  DOD  performs  comprehensive  baseline  risk  assessments  to  identify 
potential  exposure  on  receptors  such  as  humans,  and  according  to  DOD,  these  assessments  are  used 
to  justify  decisions  to  clean  up. 

®DOD’s  March  1998  “Management  Guidance  for  the  Defense  Environmental  Restoration  Program” 
details  other  management  factors  that  may  be  considered  in  sequencing  restoration  requirements. 
These  factors  include,  among  other  things,  legal  requirements;  actual  and  anticipated  funding 
availability;  and  acceptability  of  the  action  to  regulators,  tribes,  and  public  stakeholders.  DOD’s  primer 
describes  the  relative  risk  site  evaluation  process,  and  the  Qualify  Assurance  Plan  defines  objectives 
for  relative  risk  site  evaluation  data  as  established  by  the  Environmental  Security  office. 
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September  1996,  concerned  that  a  high  proportion  of  cleanup  sites  was 
justified  as  “high  relative  risk,”  the  House  Committee  on  National  Security 
requested  that  we  review  the  relative  risk  process.®  In  June  1997,  the 
Senate  Committee  on  Armed  Services  questioned  the  credibility  of  dod’s 
risk-based  approach  and  the  degree  to  which  it  facilitates  the 
establishment  of  legitimate  funding  priorities.  Because  of  this  concern,  the 
Committee  directed  dod  to  define  the  elements  of  a  relative  risk  site 
evaluation,  develop  uniform  guidance  for  evaluations  and  ranking,  and 
ensure  consistent  application  of  the  guidance. 


Results  in  Brief 


dod’s  Environmental  Security  office  provided  guidance  in  August  1997  on 
its  relative  risk  site  evaluation  process  to  defense  components  by  updating 
its  1994  Relative  Risk  Site  Evaluation  Primer  and  publishing  a  Quality 
Assurance  Plan.  The  principal  change  in  the  updated  primer  was  to 
establish  new,  DOD-wide  criteria  for  defining  low-risk  sites.  Defense 
components  are  to  provide  site  evaluation  results  to  dod  semi-annually. 
The  Environmental  Security  office  issued  the  Quality  Assurance  Plan  to 
help  ensiore  the  integrity  of  data  that  is  reported  to  dod  managers  and 
Congress.  The  plan  requires  defense  components  to  ensure  that  data  are 
credible,  auditable,  accoimtable,  and  consistent  and  to  follow  specific  data 
verification  and  reporting  procedures.  For  example,  the  plan  requires 
defense  components  to  review  the  process  used  to  derive  relative  risk  site 
evaluations  for  accuracy  and  consistency,  and  report  the  results  to 
appropriate  organizational  elements. 

DOD  used  the  relative  risk  site  evaluation  process  as  one  of  several  key 
factors  for  making  site  funding  priority  decisions.  Our  analysis  of  site 
evaluation  data  being  reported  by  the  defense  components  showed  that 
over  99  percent  of  the  reported  categorizations  were  consistent  with  dod’s 
criteria.  However,  while  consistent  with  the  criteria,  we  did  note  two 
categories  that  dod  identifies  as  high  relative  risk  and  evaluation  not 
required,  account  for  two-thirds  of  the  sites  and  three-fourths  of  the 
estimated  $15  billion  completion  costs  at  the  about  6,000  sites  we 
analyzed.  The  high  relative  risk  category  contains  1,622  sites  with  an 
estimated  $9  billion  cost  to  complete,  and  the  evaluation  not  required 
category  contains  2,584  sites,  with  an  estimated  $2.2  bfilion  cost  to 
complete.  Also,  sites  in  the  evaluation  not  required  category  may  have  high 
to  low  or  no  known  levels  of  contamination,  and  costly  long-term  remedial 
actions  or  no  anticipated  costs.  With  so  many  sites  in  these  categories,  the 


®Environinental  Protection:  Information  Used  for  Defense  Environmental  Management 
(GAO/NSIAD-97-135,  June  11, 1997). 
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designations  are  not  as  helpful  as  they  could  be  to  managers  as  one  of  the 
key  pieces  of  information  they  use  in  making  cleanup  priority  decisions 
among  competing  projects.  They  are  also  not  as  helpful  as  they  could  be  in 
assessing  the  status  of  sites  not  required  to  be  evaluated.  We  foxmd  that 
officials  at  the  installations  we  visited  were  differentiating  further  within 
the  categories  to  aid  in  their  decision-making  processes. 


DOD  Actions  to 
Provide  Uniform 
Guidance  and  Apply 
the  Relative  Risk  Site 
Evaluation  Process 


dod’s  Environmental  Security  office  has  taken  steps  to  provide  uniform 
relative  risk  site  evaluation  guidance  to  defense  components.  The  office 
updated,  in  August  1997,  its  1994  Relative  Risk  Site  Evaluation  Primer  to 
establish  new,  agencywide  criteria  for  categorizing  low-risk  sites.  At  that 
time,  the  office  also  published  a  new  Quality  Assurance  Plan  to  help 
ensure  data  integrity  in  the  application  of  the  relative  risk  site  evaluation 
process.^  As  a  result  of  the  updates  to  the  primer  and  the  establishment  of 
the  Quality  Assurance  Plan,  defense  components  updated  fiscal  year  1996 
and  1997  relative  risk  data.  The  updated  data  showed  changes  of  pathway 
or  receptor  data,  or  changes  of  20  percent  or  greater  in  reported 
contamination  levels,  for  about  2,100  of  the  6,000  sites  considered  in  our 
review.  Our  analyses  of  data  submitted  for  dod’s  1997  annual  report 
showed  that  over  99  percent  of  the  defense  components’  relative  risk 
categorizations  were  consistent  with  the  Environmental  Security  office’s 
most  recent  criteria,  dod  corrected  the  data  for  the  other  1  percent  (92 
sites)  where  we  found  that  categorizations  were  not  consistent  with  the 
reported  data.  Officials  fi"om  the  Environmental  Security  office  stated  that 
they  included  the  revised  data  in  the  current  fiscal  year  1997  annual  report 
to  Congress.® 


About  1,600  of  the  6,000  sites  changed  relative  risk  rankings  from  one  year 
to  the  next  because  of  changes  in  contamination  data  or  site  status  and/or 
because  of  the  establishment  of  a  new  threshold  below  which  sites  must 
be  ranked  as  low  relative  risk.  Table  1  shows  the  aggregate  changes  from 
fiscal  year  1996  to  1997  that  affected  1,637  of  the  6,016  sites  that  were 
common  to  both  years. 


^DOD’s  primer  describes  the  relative  risk  site  evaluation  process,  and  the  Quality  Assurance  Plan 
defines  objectives  for  relative  risk  site  evaluation  data  as  established  by  the  Environmental  Security 
office. 

®In  discussions  with  Environmental  Security  officials,  they  stated  that  the  services  and  Environmental 
Security  had  discovered  and  corrected  many,  if  not  all,  of  the  errors  we  had  identified  during  DOD’s 
quality  assurance  process. 
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Table  1 :  Reported  Changes  in  Site 
Evaiuation  Categories  (fiscai  years 
1996-97) 

Category 

Sept.  30, 
1996 

Number 
of  sites 

Sept.  30, 
1997® 

Number 
of  sites 

1996-97 

Net 

change 

1996-97 

Decreases 

1996-97 

increases 

High 

1,830 

1,594 

-236 

443 

207 

Medium 

626 

674 

48 

218 

266 

Low 

779 

645 

-134 

369 

235 

Not  evaluated 

932 

553 

-379 

506 

127 

Not  required 

1,848 

2,549 

701 

101 

802 

Total 

6,015 

6,015 

0 

1,637 

1,637 

^At  the  time  of  our  analysis,  Environmental  Security  officials  stated  that  they  considered  the  data 
as  draft  input  to  the  annual  report  to  Congress  on  environmental  restoration. 

Note:  This  table  does  not  include  sites  that  the  Environmental  Security  office  removed  from  the 
fiscal  year  1996  data  or  added  to  the  fiscal  year  1997  data.  For  the  97  high-cost  installations  in 
our  review,  DOD  provided  data  on  6,088  sites  for  fiscal  year  1996  and  6,279  sites  for  fiscal 
year  1997;  6,015  sites  were  common  to  both  years. 


New  Criteria  for  Low  in  August  1997  Environmental  Security  officials  revised  the  Relative  Risk 

Relative  Risk  Site  Evaluation  Primer  to  provide  more  specific  guidance  for  categorizing 

sites.  The  only  substantive  change  to  the  primer  was  the  addition  of  a 
requirement  that  media  (water,  soil,  and  sediment)  with  a  numeric  value 
called  a  “contaminant  hazard  factor”  below  0.005  be  assigned  to  the  low 
relative  risk  category.®  The  Environmental  Security  office  made  this 
change  to  ensure  that  the  defense  components  assign  to  the  low  relative 
risk  category  sites  with  reliable  analytical  data  that  are  within 
measurement  ranges  normally  foimd  in  noncontaminated  areas 
surrounding  a  contamination  site.^“  Although  only  five  sites  actually 
changed  categories  from  fiscal  year  1996  to  1997  as  a  result  of  the  new 
criteria,  media  ranking  changes  that  occurred  fi’om  one  year  to  the  next 
show  that  the  site  data  appeared  to  be  consistent  with  dod’s  overall 
criteria  for  assigning  relative  risk  categories.  “  For  fiscal  year  1997,  dod 


®The  contaminant  hazard  factor  is  a  ratio  that  compares  contamination  levels  with  goals  that  DOD 
calls  comparison  values. 

^°Prior  to  the  implementation  of  the  revised  primer,  factors  below  0.005  and  within  these  measurement 
ranges  could  be  categorized  high  or  medium.  Officials  believe  that  assigning  such  a  limit  will  help  them 
to  more  accurately  identify  appropriate  relative  risk  categories. 

^^We  analyzed  data  reported  to  the  Environmental  Security  office  by  the  defense  components  after 
data  were  screened  by  Environmental  Seciuity  officials.  We  verified  relative  risk  site  evaluation 
categories  and  supporting  data  only  at  the  six  locations  we  visited.  In  addition,  defense  components 
dropped  some  sites  fi*om  their  inventory,  for  example,  Anniston  Army  Depot,  one  of  the  site  locations 
we  visited.  Based  on  additional  data,  managers  deleted  this  previously  high  risk  site  with  an  estimated 
cost  to  complete  of  $118.5  million. 
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reported  that  184  of  6,279  sites  had  contamination  levels  meeting  the  new 
criteria  as  low  relative  risk.  Table  2  shows  the  five  sites  that  changed 
categories. 


Table  2:  Sites  Where  the  Reported 
Overall  Relative  Risk  Category 
Changed  as  a  Result  of  New  DOD 
Criteria  in  1997 

Site 

Fiscal  year 

1996  relative 
risk  category 

Fiscal  year 

1997  relative 
risk  category 

Underground  Storage  Site  00009,  Marine 

Corps  Air  Station,  Cherry  Point,  North  Carolina 

High 

Low 

Site  000005,  Marine  Corps  Air  Station,  El  Toro, 
California 

Medium 

Low 

Site  000007,  Naval  Air  Station,  Moffett  Field, 
California 

High 

Low 

Site  00043,  Naval  Air  Station,  Patuxent  River, 
Maryland 

Medium 

Low 

Site  00048,  Naval  Coastal  Station,  Stockton, 
California 

Medium 

Low 

Note:  Our  review  of  the  reported  data  showed  that  the  contamination  levels  for  each  of  the  five 
sites  either  did  not  change  significantly  from  one  year  to  the  next  or  did  not  change  at  all. 


Relative  Risk 

Categorizations  Consistent 
With  DOD  Criteria 


Specifically,  our  review  of  the  reported  data  showed  that  28  sites  had 
contamination  levels  below  dod’s  threshold  for  low  relative  risk  but  were 
categorized  as  medium  or  high  relative  risk.  We  brought  these  sites  to  the 
Environmental  Security  office’s  attention  during  our  review.  Officials 
subsequently  provided  the  final  data  for  the  sites  in  question,  which 
revised  data  for  26  of  the  28  sites  and  adjusted  the  relative  risk  evaluation 
categories  for  the  remaining  3  sites.  We  also  noted  that  another  67  sites 
had  overall  relative  risk  categories  and/or  individual  media  rankings  that 
did  not  appear  to  be  supported  by  the  reported  data.  Of  these,  61  had 
incorrect  overall  relative  risk  categorizations,  and  6  appeared  to  have 
correct  overall  relative  risk  categorizations  despite  errors  in  supporting 


Our  review  of  data  submitted  for  dod’s  1997  annual  report  showed  that 
over  99  percent  of  the  defense  components’  updated  relative  risk 
categorizations  were  consistent  with  the  Environmental  Security  office’s 
most  current  criteria.  Furthermore,  before  dod  published  its  most  recent 
annual  report  to  Congress,  dod  corrected  data  for  the  1  percent  (92  sites) 
where  we  found  that  categorizations  were  not  consistent  with  the  reported 
data. 
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media  rankings.  Environmental  Security  officials  subsequently  provided 
us  with  supplemental  spreadsheets  showing  that  the  final  data  had  been 
corrected  by  either  changing  the  media  or  overall  relative  risk  site 
evaluation  categorizations. 


Usefulness  of  Relative 
Risk  Evaluation 
Process  for 
Categorizing  Sites 


Our  review  of  the  reported  fiscal  year  1996  and  1997  relative  risk 
evaluation  data  for  over  6,000  sites  shows  that,  with  some  minor 
exceptions,  the  relative  risk  site  evaluation  process  enabled  the  defense 
components  to  categorize  sites.  However,  two  categories  account  for 
two-thirds  of  the  sites  and  three-fourths  of  the  estimated  completion  cost: 
the  high  relative  risk  category  (1,622  sites  with  $9  billion  cost  to  complete) 
and  evaluation  not  required  category  (2,584  sites  with  an  estimated 
$2.2  billion  cost  to  complete).  Having  such  a  large  munber  of  sites  in  these 
two  categories  in  comparison  with  the  other  categories,  raises  questions 
as  to  how  useful  the  categorizations  are  to  managers  in  assessing  relative 
risks.  We  observed,  however,  that  officials  at  the  installations  we  visited 
were  attempting  to  further  differentiate  within  each  of  the  two  categories 
to  aid  in  the  decision-making  process. 


\^^de  Range  of  dod’s  Relative  Risk  Site  Evaluation  Primer  states  that  for  all  environmental 

Characteristics  in  High  media,  three  factors  (contaminant  hazard  factor,  migration  pathway 

Relative  Risk  Category  factor,  and  receptor  factor)  are  used  in  determining  a  site’s  overall  relative 

risk  evaluation  categories  of  high,  medium,  or  low  based  on  27  possible 
different  ways  these  factors  can  be  related  to  each  other.  The  high  relative 
risk  categorizations  can  be  given  based  on  a  wide  variety  of 
characteristics;  8  out  of  27  possible  different  combinations  of  contaminant 
levels,  pathways,  and  receptors  can  be  categorized  as  high,  and  the  level  of 
contamination  can  range  from  minimal  to  significant.^^  In  fiscal  year  1997, 
about  55  percent  (1,622)  of  the  2,964  high,  medium,  and  low  relative  risk 
sites  fell  in  the  high  relative  risk  category.^'*  These  sites  have  an  estimated 
cleanup  completion  cost  of  over  $9  billion  of  the  reported  $15  billion  total 
to  complete  cleanup  at  sites  in  all  of  the  categories. 


^^efense  components  assign  a  ranking  of  high,  medium  or  low  to  each  affected  media  before 
assigning  an  overall  relative  risk  categorization.  In  some  cases,  the  supporting  media  rankings  were 
inaccurate,  but  the  overall  relative  risk  categorization  was  correct. 

defines  contaminant  hazard  factors  greater  than  100  as  “significant;”  2  to  100  as  “moderate;”  and 
less  than  2  as  “minimal.”  Our  analysis  showed  that  for  the  fiscal  year  1997  sites  categorized  as  high 
relative  risk,  the  contaminant  hazard  factors  for  the  four  highest  groimdwater  contamination  sites 
were  more  than  3  million  times  greater  than  values  for  the  four  lowest  sites. 

^‘*The  2,964  fiscal  year  1997  relative  risk  site  categorizations  were  as  follows:  hi^  =  1,622; 
medium  =  686;  and  low  =  656. 


Page  7 


GAO/NSIAD>99-25  Relative  Risk  Implementation 


B-279582 


Specific  characteristics  miist  be  present  for  a  site  to  be  categorized  high 
relative  risk.  Table  3  shows  the  eight  combinations  of  the  three  factors 
that  permit  categorizing  sites  as  high  relative  risk. 


Table  3:  Combinations  of 

Characteristics  Required  for  the  High 

Contamination® 

Migration  Pathway 

Receptor 

Relative  Risk  Category 

1.  Significant 

Evident 

Identified 

2.  Significant 

Evident 

Potential 

3.  Significant 

Potential 

Identified 

4.  Significant 

Potential 

Potential 

5.  Moderate 

Evident 

Identified 

6.  Moderate 

Evident 

Potential 

7.  Moderate 

Potential 

Identified 

8.  Minimal 

Evident 

Identified 

®Our  report  entitled  Environmental  Cleanup:  POP’s  Relative  Risk  Process  (GAO/NSiAD-98-79R, 
Feb.  26, 1998)  describes  DOD’s  process  for  determining  contamination  levels. 


Groupings  of  fiscal  year  1997  high  relative  risk  sites  having  similar 
reported  characteristics  of  contamination,  pathway,  and  receptor  showed 
little  change  fi:om  our  previously  reported  analyses.  The  major  difference 
between  our  June  1997  analysis  of  266  selected  high  relative  risk  sites  and 
our  current  analysis  of  1,622  fiscal  year  1997  high  relative  risk  sites  was 
that  the  percentage  of  sites  in  the  significant  category  fell  from  20  percent 
to  16  percent.  Sites  in  this  category  all  had  reported  significant 
contamination,  an  evident  pathway,  and  an  identified  receptor.  In 
commenting  on  a  draft  of  this  report,  dod  stated  that  these  percentages 
indicate  that  the  combination  of  relative  risk  and  other  management 
factors  is  working  very  well  in  sequencing  requirements  because  the 
components  are  focusing  their  efforts  on  the  high  relative  risk  sites,  and, 
within  the  high  category,  they  are  focusing  on  the  sites  with  higher  levels 
of  contamination  that  also  have  evident  pathways  and  identified  receptors. 
Figure  1  shows  that  there  has  been  little  change  in  site  categorizations 
between  fiscal  year  1996  and  1997. 
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potential  although  the  available  data  were  not  reported.^®  This  latter 
categoiy  increased  to  49  percent  for  fiscal  year  1997. 

Finally,  the  high  relative  risk  category,  which  is  disproportionately  large 
compared  with  the  medium  and  low  categories,  contains  contamination 
levels  ranging  from  highly  contaminated  to  minimally  contaminated.  To 
make  this  large  category  more  meaningful,  installation  managers  have 
ranked  further  within  the  category.  They  told  us  they  found  the  additional 
differentiation  useful  in  making  management  and  assessment  decisions. 


Range  of  Characteristics  in 
Evaluation  Not  Required 
Category 


dod’s  F*rimer  also  specifies  the  circumstances  under  which  site  evaluations 
are  not  required  and  permits  a  range  within  the  evaluation  not  required 
category  as  to  what  can  be  included.  The  categoiy  can  include  sites  that 
range  firom  little  or  no  contamination  to  sites  with  high  levels  of 
contamination.  The  sites  may  require  costly  long-term  cleanup  actions  or 
less  costly  monitoring,  continued  expenditures  under  other  programs,  or 
no  anticipated  costs,  dod  officials  stated  the  category  is  one  of  the  ways 
DOD  defines  success  in  the  restoration  program.  They  stated  that  they 
expect  this  categoiy  to  grow  as  final  remedies  are  put  in  place  and  the 
only  funding  requirements  left,  if  any,  are  for  remedial  action  operations  or 
long-term  monitoring.  The  evaluation  not  required  category  accounted  for 
41  percent  of  all  five  site  evaluation  categories  for  fiscal  year  1997.^® 


Continued  remediation  can  be  costly,  involving  milhons  of  dollars  over 
long  periods  of  time.  A  recent  Army  study  on  the  effectiveness  of  soil 
remediation  programs  and  groundwater  treatment  systems  noted  that 
some  of  the  systems  “appear  to  have  an  indefinite  operational  lifetime  with 
no  known  or  estimated  target  date  for  completion  of  remediation.”^^ 
Annual  costs  for  the  operation  and  maintenance  of  Army  remediation 
systems  are  approximately  $60  million.  One  remediation  system  at  the 
Twin  Cities  Army  Ammunition  Plant,  Minnesota,  is  planned  to  operate 
through  the  year  2080  at  a  total  estimated  cost  of  over  $300  million. 


^®These  include  contamination  levels  with  a  DOD  comparison  value  below  1.0.  In  commenting  on  a 
draft  of  this  report,  DOD  stated  some,  but  not  all,  of  its  comparison  values  are  based  on  EPA’s 
Preliminary  Remediation  Goals.  For  carcinogens,  DOD  stated  that  its  comparison  values  are  100  times 
the  corresponding  remediation  goal  value. 

^®The  five  categories  include  high,  medium,  low,  not  evaluated,  and  evaluation  not  required.  For  fiscal 
year  1997,  2,584  sites  were  categorized  as  evaluation  not  required.  The  evaluation  not  required 
category  also  accounted  for  nearly  78  percent  of  those  sites  categorized  as  either  not  evaluated  or 
evaluation  not  required. 

^^Evaluation  of  the  Effectiveness  of  Existing  Soil  Remediation  Programs  and  Groundwater  Treatment 
Systems  in  the  U.S.  Army,  U.S.  Army  Science  Board  Infrastructure  and  Environment  Issue  Group 
(Jan.  1998). 
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Our  analysis  of  fiscal  year  1997  evaluation  not  required  sites  showed  that 


•  At  Bangor  Naval  Submarine  Base,  Washington,  one  of  two  sites  that  fall 
under  the  evaluation  not  required  category  has  implemented  a  long-term 
“pump  and  treat”  remedial  action;  the  other  site  contains  high  levels  of 
contamination  requiring  regulators  to  agree  on  future  actions.  The  “pump 
and  treat”  remedial  action  is  part  of  an  estimated  $26.7  million  remediation 
effort  that  is  not  immediately  evident  in  the  reported  data,  dod’s  annual 
report  to  Congress  shows  one  evaluation  not  required  site,  but  provides  no 
data  on  continued  associated  estimated  costs. 

•  At  Camp  LeJeime,  North  Carolina,  a  bum  area  site  was  found  to  require  no 
further  action  after  a  remedial  investigation  was  completed  in  1996. 
Subsequently,  the  relative  risk  ranking  was  changed  from  high  to 
evaluation  not  required.  The  site  had  previously  been  listed  as  high  due  to 
the  presence  of  previously  detected  contaminants,  which  were  not  present 
during  the  remedial  investigation.  Additionally,  within  the  undergroimd 
storage  tank  program,  two  sites  had  been  ranked  as  low  and  medium  risk, 
respectively.  After  corrective  action  plans  were  completed.  Camp  LeJeune 
officials  discovered  that  existing  groundwater  contamination  at  both  sites 
contained  chlorinated  solvents  and  should  be  handled  under  a  separate 
program.  Therefore,  the  sites  were  administratively  closed  out  under  the 
undergroimd  storage  tank  program  (listed  as  response  complete),  the  cost 
to  complete  reduced  to  zero,  and  the  relative  risk  changed  to  evaluation 
not  required.  These  sites  were  then  transferred  to  the  installation 
restoration  program  to  address  groundwater  contamination  with  the 
relative  risk  listed  as  high  and  medium,  respectively.  The  combined  cost  to 
address  groundwater  contamination  is  estimated  at  $13.3  million. 

•  At  Norton  Air  Force  Base,  California,  one  site  has  begun  remedial  action 
that  will  include  about  10  to  15  years  of  long-term  operations  and 
monitoring  at  an  estimated  cost  of  $17  million.  Another  site  plans  to 
excavate  about  6,000  tons  of  soil.  At  the  time  of  this  report,  officials  had 
not  yet  estimated  completion  costs. 


Reporting  of  “Evaluation  dod’s  annual  environmental  restoration  report  to  Congress  and 
Not  Required”  Category  attachments  to  the  fiscal  year  1998-99  budget  submission  include  status 

information  on  relative  risk  sites  that  have  been  evaluated.  But,  for 
evaluation  not  required  sites,  the  range  of  site  characteristics  is  not  easily 
distinguishable:  the  documents  do  not  disclose  associated  ongoing  or 
completion  cost  estimates,  or  which  sites  have  ongoing  long-term 
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monitoring  or  operations.^®  For  example,  the  latest  report  to  Congress  for 
fiscal  year  1997  separately  identifies  17,124  sites  considered  to  be 
evaluation  not  required,  of  which  over  15,000  are  response  complete.^®  But 
the  report  also  separately  identifies  3,177  sites  that  are  in  or  planned  for 
long-term  operations  without  indicating  how  this  information  applies  to 
the  evaluation  not  required  sites,  dod  reported  that  such  sites  cost 
$172  million  in  planned  execution  for  fiscal  year  1998.  We  believe  that 
such  an  investment  warrants  identifying  which  evaluation  not  required 
sites  are  affected,  and  at  what  cost. 

Appendix  I  illustrates  the  range  of  site  characteristics  on  evaluation  not 
required  sites  for  three  installations  we  visited. 


Conclusion 


DOD  has  taken  actions  that,  if  implemented  properly,  should  improve  the 
accuracy  and  consistency  of  information  resulting  from  relative  risk  site 
evaluations,  dod  is  updating  data  in  accordance  with  the  Relative  Risk  Site 
Evaluation  Primer  and  the  Quality  Assurance  Plan.  However,  the  current 
site  categorizations  are  not  as  helpful  as  they  could  be  to  managers  in 
making  priority  decisions  among  competing  projects,  or  evident  to 
Congress  in  the  annual  reports.  The  criteria  used  for  determining  whether 
a  site  is  high  relative  risk  are  overly  broad  and  the  evaluation  not  required 
category  contains  sites  with  a  range  of  characteristics  that  isn’t  easily 
evident  in  reviewing  the  annual  environmental  restoration  report. 

As  the  process  works  now,  for  example,  a  site  with  potential  risk  to 
hiunans  can  receive  the  same  priority  as  a  site  with  actual  risk  to  humans. 
Similarly,  an  evaluation  not  required  site  with  no  contamination  or 
remediation  costs  is  grouped  for  reporting  with  contaminated  sites  with 
years  of  operational  costs  to  remediate.  Given  that  dod  uses  these 
categories  to  define  success  in  its  restoration  program,  it  would  seem 
appropriate  to  separately  identify,  for  example,  the  highest  relative  risk 
sites  in  the  annual  report,  and  to  identify  whether  the  “evaluation  not 
required”  sites  have  long-term  monitoring  or  operations,  along  with  their 
associated  costs. 


‘®Also,  the  annual  report  differentiates  evaluation  not  required  sites  “in  progress”  from  sites  with 
response  complete. 

^®As  reported  in  “DOD  Summary  Status”  table  B-6,  for  DOD  installations  and  formerly  used  defense 
sites. 
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Recommendation 

To  assist  managers  in  making  priority  decisions  in  the  relative  risk  site 
evaluation  process,  we  recommend  the  Secretary  of  Defense  direct  the 
Deputy  Under  Secretary  of  Defense  for  Envirorunental  Security  and/or 
service  components  to  provide  more  specific  categories  that  aid  in  priority 
setting  and  in  the  acciwate  reporting  of  the  status  of  sites. 

Agency  Comments 
and  Our  Evaluation 


In  commenting  on  a  draft  of  this  report,  dod  stated  that  it  does  not  concm 
with  our  recommendation.  It  stated  that  the  relative  risk  tool  was  never 
intended  to  differentiate  precisely  among  sites  or  risk  levels  and  that  it  is 
used  in  coryvmction  with  a  number  of  other  factors  to  determine  the 
priority  of  a  site  for  funding.  Consequently,  it  does  not  believe  our 
recommendation  would  necessarily  help  managers  make  better  decisions. 
DOD  stated  that  flexibility  in  decision  making  about  project  sequencing 
may  be  lost  if  managers  are  required  to  address  the  top  category  of  high 
relative  risk  sites  first.  Purttier,  it  stated  that  subdividing  the  evaluation 
not  required  category  would  also  not  improve  management  or  oversight 
for  sites  in  this  group  that  have  final  remedies  in  place  or  have  completed 
response  actions,  dod  stated  that  restoration  activities  at  these  sites  are 
limited  to  remedial  action  operations  and  long-term  monitoring,  which  it 
treats  as  non-discretionaiy  cost  commitments,  not  subject  to  prioritization. 
DOD  stated  that  regulatory  agencies  and  the  public  would  be  incensed  if  it 
delayed  action  at  remedial  sites  in  the  evaluation  not  required  category 
based  on  a  presumed  priority,  rather  than  fulfill  its  commitments.  Finally, 
DOD  stated  that  implementing  our  recommendation  would  not  benefit  dod’s 
program  but  would  actually  be  detrimental  because  it  would  require  a 
reclassification  of  existing  sites  causing  restructuring  and  expansion  of 
programming  and  budgeting  data. 


We  continue  to  believe  that  implementing  our  recommendation  would  be  a 
useful  aid  in  program  management  and  congressional  oversi^t.  As  we 
pointed  out  in  our  report,  officials  at  installations  we  visited  are  already 
differentiating  further  within  the  categories  to  aid  them  in  their 
decision-making  process.  We  added  information  to  our  report  to  make  it 
clear  that  relative  risk  is  only  one  of  a  number  of  available  factors. 
However,  the  remainder  of  dod’s  position  does  not  accurately  characterize 
our  recommendation.  The  intent  of  our  recommendation  is  to  provide 
more  specific  relative  risk  categories,  and  would  not  limit  flexibility,  since 
neither  the  current  system  nor  our  recommendation  requires  sites  to  be 
funded  strictly  in  relative  risk  order.  Also,  our  recommendation  does  not 
suggest  the  interruption  or  delaying  of  remedial  actions.  Such  actions,  if 
taken,  would  be  a  natural  part  of  the  decision-making  process.  Finally,  we 
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recognize  that  the  recommendation  could  result  in  sites  being  reclassified, 
but  it  would  not  necessitate  delays.  Reclassification  is  a  normal  part  of  the 
relative  risk  site  evaluation  process  and  is  provided  for  in  dod  program 
guidance.  For  example,  as  noted  in  our  report,  1,637  of  6,015  sites  changed 
categories  fi*om  1996  to  1997. 

DOD  also  suggested  technical  comments  which  we  incorporated  where 
appropriate,  dod’s  comments  and  our  detailed  evaluation  are  included  in 
appendix  II. 


Scope  and 
Methodology 


To  determine  the  extent  to  which  dod  has  issued  uniform  relative  risk  site 
evaluation  guidance  to  defense  components,  we  interviewed  and  reviewed 
policy  and  procedural  documents  from  officials  at  dod’s  Office  of  the 
Deputy  Under  Secretary  of  Defense  for  Environmental  Security;  the  Army, 
Navy,  Air  Force,  and  Defense  Logistics  Agency  headquarters;  and  selected 
defense  component  field  offices.  To  address  how  defense  mstallations 
applied  the  relative  risk  site  evaluation  process,  we  visited  and  obtained 
information  from  officials  at  dod’s  Office  of  the  Deputy  Under  Secretary  of 
Defense  for  Environmental  Security  and  selected  defense  component  field 
offices.  We  also  requested  and  analyzed  fiscal  year  1996-97  data  on  over 
6,000  sites  for  97  high-cost  installations  identified  in  our  June  1997  report. 
The  high-cost  installations  identified  in  our  June  1997  report  were 
reported  in  dod’s  fiscal  year  1995  annual  report  to  Congress.  Each 
installation  accounted  for  more  than  $20  million  of  planned  funding  during 
fiscal  years  1996-98  or  more  than  $100  million  of  projected  costs  from  1996 
to  completion. 

To  determine  whether  the  relative  risk  site  evaluation  process  provided 
data  that  enabled  the  defense  components  to  categorize  sites  and  prioritize 
work,  we  analyzed  the  fiscal  year  1996-97  data  on  the  over  6,000  sites  for 
the  97  high-cost  installations  identified  in  our  June  1997  report,  dod 
extracted  the  data,  which  are  input  from  the  defense  components,  from 
the  environmental  Resource  Management  Information  System  database. 
The  data  were  provided  to  us  incrementally  from  November  1997  to 
January  1998.  We  relied  on  the  accuracy  of  dod  and  service  data  in 
conducting  our  analysis  and  selectively  verified  contamination  data  for 
specific  projects  but  did  not  verify  overall  database  accuracy.  We  did  not 
assess  installation  judgments  regarding  pathways  and  receptors  of 
individual  sites.  We  visited  and/or  obtained  information  on  the  relative  risk 
site  evaluation  process  at  the  following  nulitaiy  installations,  which 
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included  installations  in  addition  to  the  97  high-cost  installations  just 
identified: 

Army 

Anniston  Army  Depot,  Alabama 
Navy 

Engineering  Field  Activity,  Northwest,  Naval  Facilities  Engineering 
Command,  Bangor  Submarine  Base,  Washington 
U.S.  Marine  Corps,  Camp  LeJeune,  North  Carolina 

Air  Force 


Kirtland  Air  Force  Base,  New  Mexico 
Norton  Air  Force  Base,  California 
Tiirker  Air  Force  Base,  Oklahoma 

For  this  report,  we  analyzed  reported  data  before  (fiscal  year  1996)  and 
after  (fiscal  year  1997)  dod’s  quality  control  updates  for  all  of  the  over 
6,000  sites  at  the  97  high-cost  installations  considered  in  our  Jime  1997 
report.  The  sites  accoimt  for  about  $16  billion  of  dod’s  estimated  total 
remaining  cleanup  costs  of  $27  billion.^® 

We  conducted  our  analysis  fi-om  November  1997  to  May  1998  in 
accordance  with  generally  accepted  government  auditing  standards. 


“Cost  estimates  in  DOD’s  database  are  reported  for  individual  sites  at  an  installation.  Although  we 
could  not  verify  the  estimates  for  each  site,  we  noted  that  the  total  estimates  for  each  installation 
generally  agreed  with  the  totals  reported  in  DOD’s  fiscal  year  1996  annual  report  to  Congress. 


Page  15 


GAO/NSIAD<99-25  Relative  Risk  Implementation 


B-279582 


We  are  sending  copies  of  this  report  to  other  appropriate  congressional 
committees;  the  Secretaries  of  Defense,  the  Army,  the  Navy,  and  the  Air 
Force;  the  Commandant,  U.S.  Marine  Corps;  and  the  Directors,  Office  of 
Management  and  Budget  and  Defense  Logistics  Agency.  We  will  also  make 
copies  available  to  others  on  request. 


David  R.  Warren,  Director 
Defense  Management  Issues 
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Characteristics  of  Sites  Designated 
Evaluation  Not  Required  at  Three 
Installations 


Site 

Relative  risk  category 

Fiscal  year  1 996  Fiscal  year  1 997 

Status 

Bangor  00204 

H 

R 

Remedial  actions  implemented;  in  long-term  operation 

Bangor  00025 

H 

R 

Remedial  actions  Implemented;  long-term  monitoring 

Norton  WT007 

R 

R 

Final  decision  document  issued;  no  further  response  action 
required 

Norton  SI  011 

R 

R 

Final  decision  document  issued;  no  further  response  action 
required 

Norton  TU015 

R 

R 

Final  decision  document  issued;  no  further  response  action 
required 

Norton  SS018 

R 

R 

Final  decision  document  issued;  no  further  response  action 
required 

Norton  SP009 

R 

R 

Final  closure  report  issued 

Norton  WR020 

R 

R 

Final  basewide  radionuclide  characterization  report  issued;  no 
further  actions  anticipated 

Norton  CG097 

R 

R 

Remedial  action  Implemented;  10-15  years  long-term 
operations  and  monitoring  anticipated 

Norton  DA096 

R 

R 

Closure  report  issued;  remedial  action  complete;  long-term 
monitoring  required 

Norton  DP003 

R 

R 

No  further  response  action  decision  document  signed 

Norton  DP004 

R 

R 

No  further  response  action  decision  document  signed 

Norton  ID022 

R 

R 

Final  closure  report  issued;  no  removal  action  required 

Norton  AOC 

N 

R 

Remedial  actions  implemented;  closure  report  in  regulatory 
review 

Norton  SA017 

N 

R 

Remedial  actions  implemented;  long-term  monitoring 

Norton  DP012 

N 

R 

Deed  restriction  placed  on  the  use  of  contaminated  parcels 

Norton  LF010 

N 

R 

Deed  restriction  placed  on  the  use  of  contaminated  parcels 

Norton  SS008 

H 

R 

Remedial  actions  implemented;  site  closed 

Norton  SI 001 

H 

R 

Remedial  actions  implemented;  long-term  monitoring 

Norton  AT005 

H 

R 

Remedial  actions  implemented;  additional  excavation  pianned 

Norton  DP014 

H 

R 

Final  closure  report  issued;  removal  action  complete 

Norton  WT013 

H 

R 

Final  closure  report  issued;  removal  action  complete 

Norton  SA019 

L 

R 

Remedial  actions  Implemented — deed  restriction 

Norton  TV021 

N 

R 

Final  closure  report  prepared;  additional  contamination 
detected 

Norton  SI  01 6 

N 

R 

Remedial  actions  implemented;  long-term  monitoring 

LeJeune  UST  000047 

M 

R 

Corrective  action  plans  completed;  continuing  work  under 
Installation  restoration  program  as  site  00091 

(continued) 
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Relative  risk  category 

Site 

Fiscal  year  1996 

Fiscal  year  1997 

Status 

LeJeune  UST  000023 

L 

R 

Corrective  action  plans  completed;  CERCLA  investigation  is 
underway  under  a  separate  installation  restoration  program; 
site  name  changed  to  site  00073® 

LeJeune  00016 

H 

R 

Remedial  investigation  completed  and  record  of  decision 
signed:  no  further  action  required 

Note:  Categories  are  as  reported  by  DOD's  Environmental  Security  office-”H=high  relative  risk; 
L=low  relative  risk;  R=:not  required;  and  N=not  evaluated. 


^LeJeune  site  00073,  with  an  estimated  completion  cost  of  $1 1 .3  million,  is  also  reported  in  DOD’s 
fiscal  year  1997  data  as  a  high  relative  risk  site. 
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Note:  GAO  comments 
supplementing  those  in  the 
report  text  appear  at  the 
end  of  this  appendix. 


See  p.  13. 


See  comment  1, 


See  comment  2. 


See  comment  3. 


See  comment  4. 


ACQUtSmON  AND 
TECHNOLOGY 


OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 

3000  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-3000 

0  A  SEP  ^938 


Mr.  David  R.  Warren 

Director,  Defense  Management  Issues 

National  Security  and  International  Affairs  Division 

U.S.  General  Accounting  Office 

Washington,  D.C.  20548 


Dear  Mr.  Warren: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  General  Accounting 
Office  (GAO)  draft  report,  ‘"ENVIRONMENTAL  CLEANUP:  DoD’s  Implementation  of 
the  Relative  Risk  Site  Evaluation  Process,”  dated  August  3,  1998  (GAO  Code 
709320/OSD  Case  1664). 

As  indicated  in  the  DoD’s  oral  comments  to  the  draft  report,  the  Department  does 
not  concur  with  the  recommendation  that  the  DoD  provide  more  specific  categories  for 
the  “high”  relative  risk  and  “evaluation  not  required”  categories  included  in  the  Relative 
Risk  Site  Evaluation  (RRSE)  Framework.  GAO  asserts  that  this  would  aid  priority 
setting  and  more  accurate  reporting  of  the  status  of  sites. 

Relative  risk  is  a  tool  designed  to  allow  managers  to  place  sites  in  three  broad 
categories  of  “high”,  “medium”  and  “low.”  It  was  never  intended  to  differentiate 
precisely  among  sites  or  risk  levels.  The  results  of  relative  risk  site  evaluations  are  used 
in  conjunction  with  statutory  and  regulatory  status  of  a  particular  installation  or  site, 
program  goals,  public  stakeholder  concerns,  and  economic  factors  to  determine  the 
priority  of  a  site  for  funding.  Consequently,  attempting  to  subdivide  the  “high”  category, 
as  GAO  recommends,  would  not  necessarily  help  managers  make  better  decisions. 

Likewise,  subdividing  the  “evaluation  not  required”  category  would  also  not 
improve  management  or  oversight  of  the  sites  in  this  group  that  have  final  remedies  in 
place  or  have  completed  response  actions.  Restoration  activities  at  these  sites  are  limited 
to  “remedial  action  operations”  and  “long-term  monitoring”,  which  DoD  treats  as  non¬ 
discretionary  cost  commitments,  not  subject  to  prioritization.  It  is  DoD’s  view  that 
neither  the  Comprehensive  Environmental  Response,  Compensation  and  Liability  Act  nor 
the  National  Contingency  Plan  contemplated  interruption  of  a  selected  remedial  action 
until  the  goals  for  the  remedial  action  are  achieved.  Regulatory  agencies  and 
communities  expect  that  remedial  actions  will  operate  continuously  until  cleanup  goals 
are  achieved.  Suspending  operations  would  undermine  DoD’s  relationship  with 
surrounding  communities  and  erode  our  relationship  with  regulators. 


Environmental  Security 


Defending  Our  Future 
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See  comment  5. 


See  comment  6. 


See  comment  4. 


Implementing  this  recommendation  would  not  benefit  DoD’s  program,  but  would 
actually  be  detrimental  For  example,  once  new  criteria  for  the  categories  were 
established,  sites  would  have  to  be  re-classified.  Programming  and  budgeting  data  would 
have  to  be  restructured  and  expanded,  and  new  program  goals  and  guidance  would  have 
to  be  developed.  More  importantly,  flexibility  in  decision  making  about  sequencing  of 
projects  may  be  lost  if  managers  are  required  to  address  the  top  category  of  high  relative 
risk  sites  first,  rather  than  being  allowed  to  factor  other  management  considerations  into 
the  decision  process.  Regulatory  agencies  and  the  public  would  be  incensed  if  DoD 
delayed  action  at  remedial  sites  in  the  “evaluation  not  required”  category  based  on  a 
presumed  priority,  rather  than  fulfilled  its  commitments.  In  summary,  GAO’s 
recommendation  would  have  a  disruptive  effect  on  a  mature  program  that  is  well  on  its 
way  to  meeting  Defense  Planning  Guidance  and  other  established  goals. 

More  detailed  comments  are  enclosed. 


Very  truly  yours, 


Enclosures 
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See  p.  13. 


See  comment  7. 


GAO  DRAFT  REPORT  -  DATED  AUGUST  3, 1998 
(GAO  CODE  709320)  OSD  CASE  1664 

“ENVIRONMENTAL  CLEANUP:  DoD’s  Implementation  of  the  Relative 
Risk  Site  Evaluation  Process” 

RECOMMENDATION 

RECOMMENDATION;  To  assist  managers  in  making  priority  decisions  in  the 
relative  risk  site  evaluation  process,  the  GAO  recommended  that  the  Secretary  of 
Defense  direct  the  Deputy  Under  Secretary  of  Defense  for  Environmental  Security 
and/or  service  components  to  provide  more  specific  categories  that  aid  priority 
setting  and  more  accurate  reporting  of  the  status  of  sites. 

DoD  Response:  Nonconcur.  DoD  believes  that  this  recommendation  reflects  a 
basic  misunderstanding  on  GAO’s  part  as  to  how  the  relative  risk  site  evaluation 
framework  is  used  by  DoD,  The  actual  funding  priority  for  a  site  is  determined 
after  relative  risk  site  evaluation  results  are  combined  with  other  important 
considerations  (e.g.  statutory  and  regulatory  status  of  a  particular  installation  or 
site,  program  execution  considerations,  public  stakeholder  concerns  and  economic 
factors).  In  that  relative  risk  is  not  the  sole  consideration,  subdividing  the  “high” 
category  and  expanding  the  applicability  of  relative  risk  to  “remedial  action 
operations”  and  “long  term  monitoring”  requirements  now  in  the  “evaluation  not 
required”  category  would  not  help  managers  made  better  decisions.  In  fact,  it 
would  limit  the  flexibility  of  managers  to  sequence  projects  and  have  other 
detrimental  effects  on  the  program  such  as  delaying  remedial  action  (assuming 
that  DoD  has  the  authority  to  prioritize  remedial  action  operations  and  long  term 
monitoring  requirements),  requiring  programming  and  budget  data  restructuring 
and  additional  data  collection,  upsetting  an  effective  system  for  identifying 
requirements,  and  disrupting  a  mature  program  that  is  well  on  its  way  to  meeting 
Defense  Planning  Guidance  and  other  goals. 


Attachment  to  Memo — GAO 
Draft  Report — OSD  Case  1664 
Page  1  of  1 
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See  comment  8. 


Now  on  p.  1 . 
See  comment  8. 
See  comment  8. 


Now  on  p.  2. 
See  comment  8. 


See  comment  8. 


Now  on  p.  2. 
See  comment  8. 

See  comment  8. 
See  comment  8. 


Now  on  p.  3. 
See  comment  8. 


GAO  DRAFT  REPORT  -  DATED  AUGUST  3, 1998 
(GAO  CODE  709320)  OSD  CASE  1664 

“ENVIRONMENTAL  CLEANUP:  DoD’s  Implementation  of  the  Relative 
Risk  Site  Evaluation  Process” 

Detailed  Comments 

1.  Page  1.  “INTRODUCTION” 

First  sentence.  Insert  the  following  after  sentence,  firom  p.  3,  line  7:  “  DoD 
estimates  it  will  spend  about  $27  billion  to  complete  cleanup  on  contaminated  sites 
beginning  in  fiscal  year  1998.  including  annual  estimates  for  the  First  six  years  and 
summary  totals,  where  applicable,  well  into  the  next  century.”  Reason:  Flow  of  report; 
important  point  to  make  early  in  report. 

2.  Page  2.  “BACKGROUND” 

First  paragraph,  second  sentence.  Insert  the  word  “relative”  between  low  and 
risk.  Reason:  Accuracy. 

First  paragraph,  third  sentence.  Recommend  rewording  as  follows:  “DOD 
does  not  require  relative  risk  site  evaluations  for  sites  that  only  require  building 
demolition/debris  removal  or  contain  abandoned  ordnance,  have  final  cleanup  remedies 
in  place,  or  are  “response  complete”  (require  no  further  cleanup  action).”  Reason:  For 
completeness,  add  full  list  of  criteria  for  DoD  sites  as  provided  in  the  DERP  Management 
Guidance. 

First  paragraph,  fourth  sentence.  Recommend  rewording  the  sentence  as 
follows:  “Not  evaluated  sites,  sites  that  do  not  require  evaluation,  and  sites  with  risk 
characterizations  are  reported  in  the  Defense  Environmental  Restoration  Program  annual 
reports  and  budget  justification  exhibits  provided  to  Congress.”  Reason:  As  written  this 
sentence  does  not  accurately  describe  all  the  relative  risk  site  information  provided  in  the 
annual  report  and  budget  exhibits. 

Second  paragraph,  fourth  sentence.  Change  “health  effects”  to  “exposure”. 
Reason:  Baseline  risk  assessments  are  not  so  comprehensive  as  to  identify  health  effects. 

3.  Page  3,  “BACKGROUNDS^ 

First  paragraph,  first  sentence.  Delete  the  word  “previously”. 

First  paragraph,  second  sentence.  Delete:  “However,  in  responding  to  our  July 
1997  report,”  and  begin  the  sentence  “They  have  also....”. 

First  paragraph,  third  sentence.  Delete  “Equally”  and  begin  the  sentence  with 
“Also”.  Reason:  Current  wording  inaccurately  suggests  that  DoD  has  changed  its 
statements  over  time  on  how  relative  risk  is  used.  The  changes  reflect  DoD’s  consistent 
approach. 

4.  Page  4,  “RESULTS  IN  BRIEF” 

Last  paragraph.  Recommend  that  GAO  not  apply  descriptors  such  as  the  “top 
category”  or  “bottom  category”  to  the  relative  risk  site  evaluation  categories.  This 
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Now  on  p.  3 
See  comment  9. 


See  comment  7. 


Now  on  p.  14. 
See  comment  10. 


Now  on  p.  7. 

See  comment  11. 


miscategorization  should  be  corrected  in  this  paragraph  and  throughout  the  report. 
Particularly  with  regard  to  the  category  “evaluation  not  required,”  it  is  inaccurate  to 
characterize  this  as  the  “bottom  category”  since  one  of  the  program  goals  is  to  have  all 
sites  progress  to  that  category.  “Evaluation  not  required”  corresponds  to  “remedy  in 
place”  or  “response  complete”  which  are  major  milestones  in  the  program.  Reason: 
Clarity  and  accuracy. 

5.  Page  5,  ^‘RESULTS  IN  BRIEF^^ 

First  paragraph,  last  sentence.  The  meaning  of  . officials  at  the  installations 
we  visited  were  attempting  to  make  distinctions  within  both  categories.”  is  unclear.  Were 
officials  attempting  to  make  distinctions  in  their  relative  risk  evaluations  in  these 
categories  or  were  they  following  DOD’s  management  guidance  and  using  other 
management  factors  in  conjunction  with  the  relative  risk  site  evaluation  framework  to 
help  make  prioritization  and  sequencing  decisions?  The  latter  is  expected  and 
appropriate.  The  former  sounds  like  a  misunderstanding  by  the  GAO  representatives 
interviewing  the  officials  at  the  base  or  bases  visited.  Regarding  the  “evaluation  not 
required”  category,  one  of  the  ways  we  define  success  in  the  program  is  the  movement  of 
sites  into  this  category  because  that  means  that  final  remedies  are  in  place  for  the  site  and 
the  only  funding  requirements  left,  if  any,  are  for  remedial  action  operations  or  long-term 
monitoring.  DOD  does  not  have  the  ability,  once  a  remedy  has  been  selected  and 
constructed,  to  choose  not  to  operate  the  remedy  prior  to  achieving  the  cleanup  objectives 
or  not  to  perform  long-term  monitoring  required  to  confirm  the  success  of  the  remedy. 
Therefore,  the  relative  risk  site  evaluation  framework  does  not  apply  to  these  sites. 
Recommend  GAO  clarify  what  point  is  being  made.  Reason:  Clarity. 

6.  Page  *‘DOD  ACTIONS  TO  PROVIDE  UNIFORM  GUIDANCE  AND  APPLY 
THE  RELATIVE  RISK  SITE  EVALUATION  PROCESS” 

First  paragraph,  fourth  sentence.  The  meaning  of  this  sentence  is  unclear. 

“The  updated  data  showed  changes  of  20  percent  or  greater  in  reported  contamination 
levels,  pathway,  or  receptor  data  for  about  2,100  of  the  6,000  sites  considered  in  our  June 
1 997  report.”  What  is  the  point?  Also,  did  the  referenced  report  address  6,000  sites  or  is 
it  the  current  draft  report  that  addresses  that  number  of  sites?  Suggest  GAO  clarify  the 
point  being  made  and  its  significance.  Reason:  Clarity. 

7.  Page  11,  “USEFULNESS  OF  RELATIVE  RISK  EVALUATION  PROCESS 
FOR  CATEGORIZING  SITES’’ 

First  paragraph,  second  sentence.  Does  the  $2.2  billion  include  “not  evaluated 
sites,  as  the  first  paragraph  on  Pg.  2  mistakenly  indicates  are  in  this  category?  Also,  see 
comment  2  (first  paragraph,  fourth  sentence).  This  paragraph  seems  to  lose  sight  of  the 
fact  that  relative  risk  site  evaluation  is  a  programmatic  tool  that  is  used  in  conjunction 
with  other  management  factors  and  that  remedial  action  operations  and  long  term 
monitoring  requirements  are  commitments,  not  subject  to  prioritization  by  DOD. 
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Now  on  p.  7. 
See  comment  8. 

See  comment  8. 
See  comment  8. 


Now  on  p.  8. 

See  comment  12. 


Now  on  p.  9. 
See  comment  8. 


Now  on  p.  9. 
See  comment  8. 


8.  Page  11,  **Wide  Range  of  Characteristics  in  High  Relative  Risk  Category” 

First  paragraph,  first  sentence.  At  the  end  of  the  sentence  after  “low”  insert; 
'•based  on  27  possible  different  wavs  these  factors  can  be  related  to  each  other.”  Reason: 
Clarity. 

First  paragraph,  second  sentence.  This  sentence  is  misleading  and  should  be 
deleted  or  at  least  the  word  “wide”  should  be  deleted.  Reason:  Clarity. 

First  paragraph,  third,  fourth  sentences.  Recommend  the  following  wording 
be  substituted:  “For  example,  eight  out  of  27  possible  different  combinations  of 
contaminant  levels,  pathways  and  receptors  can  be  categorized  as  high.”  The  fourth 
sentence  beginning  with  “Contaminant  levels. should  be  deleted  because  it  is  incorrect 
as  written  and  the  point  that  the  sentence  is  trying  to  make  is  correctly  illustrated  in  Table 
3  on  page  12,  Reason:  Accuracy  and  balanced  perspective. 

9.  Pages  12-13,  “Wide  Range  of  Characteristics  in  High  Relative  Risk  Category” 

Two  sentences  that  read:  “As  shown  in  figure  1,  the  major  difference  between 
our  June  1997  analysis  of  266  selected  high  relative  risk  sites  and  our  current  analysis  of 
1,622  fiscal  year  1997  high  risk  sites  was  that  the  percentage  of  sites  in  the  “significant” 
category  fell  from  20  percent  to  15  percent.  Sites  in  this  category  all  had  reported 
significant  contamination,  an  evident  pathway,  and  an  identified  receptor.”  The 
significance  of  these  statements  is  not  clear,  however,  they  appear  to  indicate  that  the 
combination  of  relative  risk  and  other  management  factors  is  working  very  well  in 
sequencing  requirements  because  evidently,  not  only  are  the  Components  focusing  their 
efforts  on  the  high  relative  risk  sites,  but  within  the  high  category  they  are  focusing  on  the 
sites  with  higher  levels  of  contamination  that  also  have  evident  pathways  and  identified 
receptors.  Suggest  GAO  clarify  the  point  being  made.  Reason:  Clarity. 

10.  Page  13,  “Wide  Range  of  Characteristics  in  High  Relative  Risk  Category 

In  the  descriptions  of  the  two  pie  charts  of  Figure  1 :  “DOD  Sites  With  High 
Relative  Risk”  delete  the  word  “EPA’s”  and  insert  in  its  place  “DOD  comparison  value 
screening  criteria”.  Not  all  of  DOD’s  comparison  values  are  based  on  EPA’s  Preliminary 
Remediation  Goals  (PRGs)  and  for  carcinogens  DOD’s  comparison  values  are  100  times 
the  corresponding  PRG  value.  Delete  the  phrase  “  because  data  were  not  available.”  The 
statement  is  not  correct.  Data  can  be  complete  and  the  pathway  or  receptor  can  still  be 
“potential.”  For  example,  data  indicates  that  contamination  is  moving  from  the  source 
but  has  not  reached  a  receptor  yet  or  property  is  contaminated  and  undeveloped  but  can 
be  developed.  In  the  first  case,  there  are  evident  receptors  and  a  potential  pathway.  In 
the  second  case,  the  receptors  are  potential.  In  both  of  these  very  common  scenarios,  the 
data  are  complete.  Reason:  Accuracy. 

11.  Page  14,  “Wide  Range  of  Characteristics  in  High  Relative  Risk  Category’’ 

First  paragraph.  The  entire  first  paragraph  should  be  revmtten  to  read:  “As 
shown  in  figure  T  we  also  reported  in  June  1997.  that  54  percent  of  the  high  relative  risk 
sites  had  a  moderate  or  significant  contaminant  hazard  factor  with  an  evident  pathway 
and  identified  receptor.”  The  second  sentence  is  not  correct  and  should  be  deleted.  In 
order  to  do  relative  risk  site  evaluations,  all  three  factors  have  to  be  evaluated  based  on 
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Nowon  p.  10. 
See  comment  9. 

See  comment  7. 


Now  on  p.  11. 
See  comment  13. 


See  comment  8. 


Nowon  p.  11. 


available  information.  Sites  lacking  information  for  any  of  the  three  factors  are 
categorized  as  “not  evaluated”  (See  page  5,  Section  1.4  of  the  Relative  Risk  Site 
Evaluation  Primer).  For  the  corrections  to  the  third  sentence  see  Comment  8.  Reason: 
Accuracy. 

12.  Page  14,  “Wide  Range  of  Characteristics  in  ‘^Evaluation  Not  Required” 
Category” 

This  section  is  misleading  and  contains  inaccuracies.  It  seems  to  have  been 
written  with  the  intent  of  trying  to  develop  a  rationale  for  expanding  the  applicability  of 
relative  risk  site  evaluations  to  the  end  phases  of  the  program  (remedial  action  operations 
and  long  term-monitoring),  as  a  vehicle  for  prioritizing  these  requirements.  However, 
remedial  action  operations  and  long-term  monitoring  funding  requirements  are 
commitments  that  are  not  subject  to  prioritization.  Relative  risk  is  not  the  right  tool. 
These  requirements  should  be  addressed  by  using  alternative,  less  stringent  cleanup 
approaches  based  on  future  land  use;  careful  examination  of  cleanup  assumptions,  and 
application  of  innovative  technologies  in  order  to  minimize  the  associated  costs  for 
remedial  action  operations  and  long  term  monitoring.  Recommend  that  this  entire  section 
be  redrafted.  Reason:  Clarity. 

13.  Page  15,  Last  paragraph.  Replace  with  the  following: 

“At  Camp  Leieune.  NC.  a  bum  area  site  was  found  to  require  no  further  action  after  a 
Remedial  Investigation  tRH  was  completed  in  1996.  Subsequently,  the  relative  risk 
ranking  was  changed  from  “high”  to  “evaluation  not  required”.  The  site  had  previously 
been  listed  as  high  due  to  the  presence  of  previously  detected  contaminants  which  were 
not  present  during  the  RI.  Additionally,  within  the  UST  program,  two  UST  sites  had 
been  ranked  as  low  and  medium  relative  risk,  respectively.  After  Corrective. Action  Plans 
( C  APs)  were  completed,  it  was  discovered  that  existing  groundwater  contamination  at 
both  sites  contained  chlorinated  solvents  and  should  be  handled  under  a  separate 
program.  Therefore,  the  sites  were  administratively  closed  out  under  the  UST  program 
riisted  as  response  complete!,  the  cost  to  complete  reduced  to  zero,  and  the  relative  risk 
changed  to  “evaluation  not  required”.  These  sites  were  then  transferred  to  the  Installation 
Restoration  Program  to  address  groundwater  contamination  with  the  relative  risk  Listed  as 
“high”  and  “medium”  respectively.  The  combined  cost  to  address  groundwater 
contamination  is  estimated  at  $13.3  million.”  Reason:  The  paragraph  is  confusing  and 
inaccurate  as  written. 

14.  Page  16,  Footnote.  Delete  this  footnote.  Both  USTs  have  been  administratively 
closed  out  under  the  state’s  UST  program.  The  remaining  contamination  will  be 
addressed  under  the  Installation  Restoration  Program;  however,  this  is  now  stated  in  the 
above  paragraph.  Reason:  Accuracy. 

15.  Page  16-17,  “Use  of  Relative  Risk  Categories” 

Recommend  a  new  title  for  this  section.  The  current  title  seems  to  be  unrelated  to 
the  discussion  which  is  mostly  about  the  FY1997  annual  report. 
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Nowon  p.  11. 

First  paragraph,  first  sentence:  The  meaning  of  this  sentence  is  unclear.  Is  this 

See  comment  8. 

sentence  describing  the  FY1997  annual  report?  There  is  a  lot  of  relative  risk  information 
in  the  annual  report  including  statistics  and  uses  of  relative  risk  in  setting  program  goals, 
measuring  progress  and  as  a  tool  for  sequencing  requirements.  Recommend  that  this 
sentence  be  rewritten.  Reason:  Clarity. 

See  comment  8. 

First  paragraph,  second  sentence.  The  second  sentence  does  not  follow  from 

the  first  sentence.  Also,  it  is  not  evident  how  the  document  referenced  at  footnote  16 
provides  examples  of  discrepancies  underlying  relative  risk  information  provided  to 

Congress.  Did  the  committee  report  mentioned  come  out  before  or  after  the  FY1997 
annual  report  was  published?  Recommend  redrafting  the  paragraph  to  ensure  logical 
flow  from  sentence  1  to  2,  and  deleting  footnote  16.  Reason:  Flow  of  report. 

See  comment  8. 

Second  paragraph,  first  sentence:  The  report  does  in  fact  identify  that  sites  in 

“remedial  action  operations”  are  categorized  as  “evaluation  not  required”  (see  notes  on 

Tables  2  and  3,  pages  13  and  14).  Also,  the  use  of  the  term  “bottom  category”  to  describe 
“evaluation  not  required”  is  not  appropriate  considering  that  one  of  the  program  goals  is 
to  have  ail  sites  advance  into  this  category.  DOD  counts  sites  in  this  category  as  a 
success  (see  Comment  4).  Also,  is  the  $2.2  billion  estimate  just  for  97  installations  or  for 
all  the  installations  and  how  does  this  estimate  relate  to  the  $2.2  billion  estimate  on  page 

1 1  which  is  associated  with  a  different  number  of  sites?  Reason:  Accuracy. 

See  comment  8. 

Second  paragraph,  second  sentence:  As  pointed  out  in  Comment  5,  the 

statement  in  the  second  sentence  that  installation  officials  were  attempting  to  make 
distinctions  within  categories  needs  more  explanation  than  “for  their  own  purposes”. 

Reason:  Key  statements  need  support  for  credibility. 

See  comment  8. 

Third  paragraph:  What  is  GAO’s  definition  of  broad?  Appendix  I  does  not 

illustrate  a  broad  range  of  characteristics  for  the  “evaluation  not  required”  category  as 
stated  by  GAO.  All  of  the  sites  meet  established  criteria  for  the  category.  We  count  the 
reductions  in  the  number  of  sites  in  the  high,  medium,  and  low  categories  as  progress 
towards  another  program  goal  and  track  that  progress  with  a  measure  of  merit.  Reason: 

Accuracy. 

16.  Pace  17.  “CONCLUSION  and  RECOMMENDATION” 

Now  on  pp.  12-13. 

In  order  to  accurately  reflect  the  findings  stated  on  pages  9  and  10,  recommend 

See  comment  15. 

revising  the  first  sentence  in  the  “CONCLUSION”  to  read:  “DOD  has  taken  actions  that 
have  improved  the  accuracv  and  consistency  of  information  resultine  from  relative  risk 
site  evaluations.  In  fact.  99  nercent  of  the  draft  data  and  100  percent  of  the  final  data 
reviewed  was  in  accordance  with  the  Relative  Risk  Site  Evaluation  Primer  and  OA  Plan 
reauirements.” 

DOD  does  not  agree  with  the  other  conclusions  and  recommendation  in  this  report 

for  the  following  reasons: 

See  comment  7. 

•  The  conclusions  and  recommendation  reflect  a  basic 

misunderstanding  of  what  relative  risk  is  and  how  it  is  used.  GAO  does  not  seem 
to  recognize  that  the  relative  risk  site  evaluation  framework  is  not  a  risk 
assessment  and  also  is  not  the  sole  factor  used  to  prioritize  requirements.  Other 
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See  comment  1. 


management  factors  (such  as  legal  requirements,  program  execution,  stakeholder 
concerns,  and  economic  considerations)  are  used  in  addition  to  relative  risk  in 
order  to  sequence  work. 

•  In  addition,  DOD  does  not  believe  that  relative  risk  site  evaluations 
are  applicable  to  sites  that  have  progressed  to  “remedy  in  place”  or  “response 
complete”  because  the  only  requirements  left  once  these  milestones  have  been 
achieved  are  for  “remedial  action  operations”  and  “long-term  monitoring”  which 
are  viewed  as  commitments  and,  therefore;  not  subject  to  prioritization. 

•  It  is  our  view  that  neither  CERCLA  nor  the  NCP  contemplated 
interruption  of  a  selected  remedial  action  until  the  goals  for  the  remedial  action 
are  achieved.  We  do  not  believe  DOD  has  the  authority  to  suspend  operations 
prior  to  achieving  cleanup  goals.  We  also  do  not  believe  that  DOD  has  the 
authority  to  suspend  required  long-term  monitoring  necessary  to  confirm  the 
success  of  remedial  actions. 
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GAO  Comments 


The  following  are  gao’s  comments  on  the  Department  of  Defense’s  (dod) 
letter  dated  September  4, 1998. 


1.  dod’s  high  relative  risk  category  is  overly  large  in  relation  to  the  medium 
and  low  categories,  and  thus  does  not  provide  three  broad  categories.  To 
make  this  more  clear,  we  added  an  example  in  the  report  that  dod’s 
highest  contaminant  hazard  factors  were  more  than  3  million  times  greater 
than  the  lowest  ones  at  sites  within  the  high  relative  risk  category. 

2.  We  agree  that  relative  risk  is  used  in  coryunction  with  other  important 
factors,  but  dod’s  conclusion  does  not  follow.  For  those  site  rankings 
where  relative  risk  affected  managers’  decisions,  the  additional  category 
data  would  help  the  decision.  We  added  to  our  report  an  example  of  a 
previously  high  relative  risk  site  that  managers  deleted  from  inventory 
based  on  additional  data. 

3.  dod’s  comment  that  the  two  types  of  sites  are  non-discretionaiy  cost 
commitments  is  inconsistent  with  defense  components’  ongoing 
reconsideration  of  remedial  action  operations,  such  as  cited  in  our  report. 
We  agree  such  sites  are  not  subject  to  further  prioritization. 

4.  Our  recommendation  does  not  necessitate  interruption  of  remedial 
actions.  Reclassification  is  a  normal  part  of  the  relative  risk  site  evaluation 
process  and  is  provided  for  in  dod’s  Relative  Risk  Site  Evaluation  Primer. 
Also,  as  stated  in  our  report,  defense  components  are  themselves 
considering  where  such  actions  may  be  appropriate. 

6.  Although  our  recommendation  could  result  in  reclassified  sites,  such 
actions  are  a  normal  part  of  dod’s  process,  as  evidenced  by  the  over  1,600 
category  changes  from  1996  to  1997. 

6.  Our  recommendation  does  not  prohibit  managers  from  using  other 
management  considerations  in  their  decision  process.  It  simply  suggests 
that  more  specificity  within  the  two  categories  could  be  more  helpful  to 
decisionmakers. 

7.  Both  our  draft  and  final  reports  recognize  that  relative  risk  is  one  of 
several  factors  considered  by  decisionmakers.  Also,  as  we  stated  in 
addressing  dod  comments  on  our  June  1997  report,  we  recognize  that  the 
relative  risk  site  evaluation  process  is  an  initial  screening  method  that  is 
only  one  of  the  factors  considered  by  decisionmakers.  Further,  dod  fully 
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concurred  with  the  description  of  the  relative  risk  site  evaluation  process 
in  our  report  entitled  Environmental  Cleanup:  pod’s  Relative  Risk  Process 
(GAO/NSIAD-98-79R,  Feb.  26, 1998).  The  recommendation,  as  written,  is 
sufficiently  broad  as  to  allow  dod  flexibility  in  how  best  to  differentiate 
within  the  categories.  We  do  not  believe  that  better  defining  and  making 
categories  more  specific  would  limit  flexibility  because  neither  the  current 
system  nor  our  recommendation  requires  sites  to  be  funded  strictly  in 
relative  risk  order.  Thus,  the  recommendation  also  would  not  delay 
remedial  action,  require  programming  and  budget  data  restructiuing  and 
additional  data  collection,  upset  the  existing  system  for  identifying 
requirements,  or  disrupt  the  existing  program. 

8.  We  revised  our  report  to  reflect  dod’s  suggested  changes. 

9.  We  addressed  dod’s  questions.  The  evaluation  not  required  category 
may  have  high  to  low  or  no  known  levels  of  contamination,  and  costly 
long-term  remedial  actions  or  no  anticipated  costs.  Officials  at 
installations  we  visited  were  differentiating  further  within  both  the  high 
relative  risk  and  evaluation  not  required  categories  to  aid  in  their 
decision-making  processes.  We  inserted  dod’s  statement  that  it  uses  the 
evaluation  not  required  category  to  define  success  in  the  restoration 
program. 

10.  We  revised  our  report  to  state  that  the  updated  data  referred  to  the 
about  6,000  sites  considered  in  our  current  review.  This  is  simply  a 
statement  of  fact  describing  the  number  of  changes  and  their  extent,  and 
its  puipose  is  evident,  as  stated. 

11.  We  revised  our  report  to  make  it  more  clear  that  the  $2.2  billion  refers 
to  sites  in  the  evaluation  not  required  category. 

12.  The  statements  simply  reflect  the  results  of  our  analysis.  We  added 
language  to  reflect  dod’s  comment  that  it  believes  the  combination  of 
relative  risk  and  other  management  factors  is  working  well  in  sequencing 
requirements. 

13.  The  information  in  our  draft  report  was  based  on  specific  input  from 
Marine  Corps  officials.  Although  the  information  was  accurate  as  stated, 
we  included  dod’s  suggested  language. 

14.  We  modified  this  section  of  the  report  to  focus  on  reporting  of  the 
evaluation  not  required  category. 
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15.  We  revised  our  report  to  include  specific  reference  to  dod’ s  positive 
actions.  Although  our  tests  confirmed  that  components  followed 
processes  for  contamination  levels,  we  could  not  state  that  100  percent  of 
all  data  met  requirements. 
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